CRUD for automations in the API and UI - #2294
Conversation
- New endpoints on assets: POST /automations (create, validating parameters by automation type), PATCH /automations/<id> (name, cron string, activation status) and DELETE /automations/<id>. Managing automations requires the same principals that may delete the asset (account admins and consultants). - The UI automations page gets a 'New automation' modal and per-row (de)activate and delete actions, shown to users with management rights. - Creation, update and deletion logic (incl. audit log records) moved into the automations service, shared by the CLI commands and the API endpoints. Part of #2288 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rbix8k1JfeUWNXEmHEZVpX
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rbix8k1JfeUWNXEmHEZVpX
Documentation build overview
135 files changed ·
|
…elves Context: - Review of #2290 asked that automations administered through the UI (and hence the API) may only involve sensors the creating user has access to; account admin rights on the asset should not grant access to another account's sensors Change: - Work out the sensors an automation would read from and write to (forecasts: the sensor to forecast plus its regressors, and the sensor to save to; schedules: the flex-model's device sensors, and whatever the parameters refer to) - Require read access to the former and create-children (the permission for recording data through the API) on the latter, when creating via the API - The CLI creates automations without a user, and stays unrestricted Signed-off-by: F.N. Claessen <felix@seita.nl>
Context: - The endpoint description now states the sensor access rule Change: - Regenerated the specs Signed-off-by: F.N. Claessen <felix@seita.nl>
Context: - The sensor access rule for created automations needs regression coverage Change: - An account admin creating an automation on another account's sensor gets a 403 naming that sensor, and no automation is created; the same request on their own sensor still succeeds (verified to fail without the check) Signed-off-by: F.N. Claessen <felix@seita.nl>
Context: - The sensor access rule is user-facing Change: - Documented it in the forecasting feature docs, the changelog entry of #2294 and the API change log Signed-off-by: F.N. Claessen <felix@seita.nl>
Context: - Schedulers hand their results to make_schedule as (sensor, data) pairs, and those sensors are not only the flex-model's device sensors: a schedule is also recorded on a device's state-of-charge, consumption and production sensors, and on the flex-context's aggregate-consumption and aggregate-production sensors Change: - Derive a schedule's output sensors from all the fields that name where generated data goes, at any depth in the flex-model and flex-context (which schedulers deserialize themselves, so their sensor references are still raw) - Everything else the parameters refer to (e.g. price sensors and the sensors of inflexible devices, which may also live on the flex-context) counts as an input Signed-off-by: F.N. Claessen <felix@seita.nl>
Context: - The flex-context's aggregate-consumption sensor is written to, so it needs the same check as the flex-model's own sensors Change: - Posting such an automation gets a 403 that names the sensor and the action (verified to fail when only the flex-model's sensors are treated as outputs) Signed-off-by: F.N. Claessen <felix@seita.nl>
|
@BelhsanHmida — Felix reviewed the automations work and asked for one thing to land here: an automation administered through the UI (and hence the API) may only involve sensors that its creator can access. Could you review these commits, and then take the PR over again? What we contributed:
Two things to be aware of:
🤖 Generated with Claude Code |
Brings in schedule automations, the timezone and catch-up work, the sensor links and the review fixes from further down the stack. The CRUD refactor is kept: the CLI still calls create_automation and update_automation rather than building automations inline, so the services carry the logic that the base had grown there. create_automation therefore takes a timezone, and update_automation takes one too and rebases the scheduling cursor whenever it changes what is due, namely the recurrence, the timezone or reactivation. Two resolutions went further than picking a side. The forecast output scope is now validated after the access check rather than before, so that a sensor the user may not read is refused as forbidden rather than described as being outside the asset. A test that created a forecast automation without a data generator now passes one, as the base requires forecasts to have one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…olves A regressor that filters on sources deserializes into a sensor reference rather than a sensor, which collect_sensors skipped, so such a regressor was left out of the sensors an automation reads from. The access check is built on that list, so a user could set up an automation reading a sensor they cannot read themselves. Ask the forecaster instead, as it derives its input and output sensors from the same config and parameters it will run with, and already resolves sensor references. Schedules keep their own collection, as they have no data generator to ask. Displaying the sensors involved and checking access to them now share one implementation, so they cannot disagree. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
… API An automation carries the timezone its cron expression is interpreted in, and the CLI can set and change it, but the API could do neither, so every automation created through the API or the UI was stuck on the server's timezone. Both the creation and the update schema now accept a timezone, defaulting to FLEXMEASURES_TIMEZONE on creation. Also restores the OpenAPI spec's version string, which a regeneration during the merge had replaced with the locally installed version. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
…mation is allowed Creating an automation looked up or created the data source holding the forecaster configuration before checking whether the user may involve the sensors at all, so a refused request still added a data source within that request. Nothing committed in between, so this did not outlive the request, but it relied on that rather than on the order of events. The data source is now set up after the access check, which makes a refused request leave nothing behind by construction. Also records what the output sensor field list approximates, namely the sensors a scheduler returns results for at run time, and therefore how it can drift away from them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com> # Conflicts: # flexmeasures/api/v3_0/assets.py # flexmeasures/data/schemas/automations.py # flexmeasures/data/services/automations.py
Permission failures now identify an inaccessible automation dependency only by the sensor ID supplied in the request. This preserves a useful reference for the caller without confirming private sensor names across organisation boundaries. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Automation endpoint tests now use function-scoped fresh database fixtures because they create, update, and delete automations and related sensors. The permission cases also assert that forbidden responses retain the submitted sensor ID without disclosing its private name. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
The asset automations view now supplies the canonical IANA timezone choices accepted by the automation schema. Keeping the options server-side ensures the create and edit controls offer the same vocabulary that the API validates. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Managers can now choose an IANA timezone when creating an automation and edit its name, recurrence, timezone, and active state from the asset page. New automations default to the asset timezone, while the API remains responsible for validating every submitted value. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
The asset page regression test verifies that managers receive create and edit timezone fields, that creation starts from the asset timezone, and that both forms include their selected timezone in the corresponding API request. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
The automation CRUD entry now records that recurrence timezones are selectable in the user interface and uses the established organisation terminology for the people allowed to manage them. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Schedule sensor discovery now leaves creation-time schema and scheduler errors intact so the CLI and API can render their established validation responses. Stored automation resolution still wraps those failures as unknown dependencies for strict permission checks and lenient displays. Signed-off-by: Mohamed Belhsan Hmida <mohamedbelhsanhmida@gmail.com>
Manual test walkthroughCovers what CRUD adds over #2290 and #2293: administering automations through the API and the UI, You need two logins to see the interesting half:
The examples use asset 1. The page gains controls, for the right peopleAs an account admin the listing gains a New automation button and per-row Edit, 2. Create one from the UINew automation asks for the name, the type, the recurrence, the timezone it is interpreted in, 3. The same over the API# create
POST /api/v3_0/assets/242/automations
{
"name": "Daily campus forecast",
"type": "forecasts",
"cronstr": "0 6 * * *",
"timezone": "Europe/Amsterdam",
"parameters": {"sensor": 913}
}
# → 201, with the automation, its timezone and its scheduling cursor
# change the recurrence, the timezone and the activation status
PATCH /api/v3_0/assets/242/automations/<N>
{"cronstr": "30 7 * * *", "timezone": "Asia/Seoul", "active": false}
# → 200
# delete
DELETE /api/v3_0/assets/242/automations/<N>
# → 204The timezone is worth checking specifically: an automation created through the API used to be stuck Note that 4. You may only automate sensors you can accessThis is the substance of the PR. As an account admin of one organisation, try to forecast a sensor POST /api/v3_0/assets/242/automations
{"name": "Refused", "type": "forecasts", "cronstr": "0 6 * * *",
"parameters": {"sensor": 426}}HTTP 403
{
"message": "You cannot set up an automation that would read data from sensor 426, because you cannot read data from it yourself.",
"result": "Rejected",
"status": "INVALID_SENDER"
}A regressor that filters on sources is a sensor reference rather than a plain sensor, and is checked POST /api/v3_0/assets/242/automations
{"name": "Refused", "type": "forecasts", "cronstr": "0 6 * * *",
"parameters": {"sensor": 913},
"config": {"regressors": [{"sensor": 426, "source-types": ["forecaster"]}]}}HTTP 403
{
"message": "You cannot set up an automation that would read data from sensor 426, because you cannot read data from it yourself.",
"result": "Rejected",
"status": "INVALID_SENDER"
}Three things are worth confirming while you are here:
For schedules the same applies to whatever the schedule would be recorded on, including sensors the 5. The CLI stays unrestrictedflexmeasures add automation --asset 242 --name "From the CLI" --cron "0 6 * * *" --sensor 913The CLI runs without a user, so it is trusted and not subject to the check above. That is deliberate: 6. Clean upDelete anything you created, from the listing's Delete action or with |
Description
Automations can now be created, updated and deleted through the API and the UI, not only from the
CLI.
API.
[POST] /assets/(id)/automations,[PATCH] /assets/(id)/automations/(automation_id)and[DELETE] /assets/(id)/automations/(automation_id). They require the same rights as deleting theasset — account admins and consultants — which matches the automation's own access rules: whoever
may read an asset may read its automations, and whoever may delete it may change them. A
PATCHcovers the name, the recurrence, the timezone and the activation status; the parameters are
deliberately not editable, so the sensors an automation involves stay the ones its creator was
checked against.
UI. The asset's Automations page gains a creation modal and per-row actions for those users.
Only sensors the creator can access. An automation administered this way may only involve
sensors its creator can access themselves: read access to the sensors it reads from, and
create-children on the sensors it writes to, which is the permission the API already requires for
recording data on a sensor. A refused request gets a
403naming the sensor and the action. Thissits behind a flag that the API passes; the CLI creates automations without a user and stays
unrestricted.
Which sensors those are depends on the type:
config and parameters it will run with, so a regressor that filters on sources counts too even
though it is a sensor reference rather than a plain sensor.
tree are included, and the outputs are then taken from the fields that name where results are
recorded: a device's power sensor, its state of charge, consumption and production sensors, and
the flex-context's aggregates. Everything else the parameters refer to counts as an input.
Timezone through the API. An automation carries the timezone its recurrence is interpreted in.
Creation and update now accept it, so automations administered through the API or the UI are no
longer stuck on the server's timezone. Changing it rebases the scheduling cursor, exactly as the CLI
does.
Refusals leave nothing behind. The data source holding a forecaster's configuration is set up
only once the automation is allowed, so a refused request adds nothing. The check on where a
forecast may be recorded runs after the access check, so a sensor the caller may not read is refused
as forbidden rather than described as being outside the asset.
Consolidation. Creating, updating and deleting live in
flexmeasures/data/services/automations.py,so the CLI and the API share one implementation rather than each building automations inline.
documentation/changelog.rstLook & Feel
An account admin gets a New automation button and per-row Edit, Activate /
Deactivate and Delete actions:
The same page, for the same asset, as a plain user of the same organisation. The automations and
their details are still readable; nothing that would change them is offered:
Creating one asks for the type, the recurrence and the timezone it is interpreted in, and the
parameters — forecast parameters, or a schedule trigger message:
How to test
See the manual test walkthrough in the PR comments.
Coverage includes a forecast on another organisation's sensor, a forecast whose source-filtered
regressor is another organisation's sensor, a schedule aggregated onto another organisation's
sensor, and the timezone roundtrip through creation and update. Each was verified to fail without
the check it covers.
Further improvements
reading the fields that name where results go. That agrees with the scheduler's own resolution for
a device's
sensor,consumptionandproduction, and errs towards reporting more rather thanfewer. Checking outputs against what a scheduler actually returns at run time is tracked in Check the sensors a scheduler actually writes to, instead of predicting them when an automation is created #2421.
means deleting it and creating a new one, which keeps the access check honest but is blunt.
Related items
Closes #2372. Part of the automations story #2334. Stacked on #2293.
Sign-off